Skip to content

feat: replace Dropy dropdowns with custom compact grid selector in Filtering tool#1009

Merged
tariqksoliman merged 1205 commits into
NASA-AMMOS:developmentfrom
JPL-Devin:development
Jun 29, 2026
Merged

feat: replace Dropy dropdowns with custom compact grid selector in Filtering tool#1009
tariqksoliman merged 1205 commits into
NASA-AMMOS:developmentfrom
JPL-Devin:development

Conversation

@tariqksoliman

Copy link
Copy Markdown
Member

With Devin: JPL-Devin#100

Purpose

  • The value operator and group operator dropdowns in the Filtering tool used Dropy with absolute positioning, causing clipping by the header and panel top when rows are near the top of the viewport.
  • Replace Dropy entirely with a custom compact jQuery grid selector (OpGridSelector) that renders a fixed-position popup appended to <body>, completely escaping ancestor overflow clipping.
Screenshot 2026-06-29 104659

devin-ai-integration Bot and others added 30 commits June 9, 2026 21:23
Two bugs in the previous convergence fix:

1. _grid_convergence used atan2(x, -y) unconditionally, but for
   south-pole stereo the sign should be +1 (north is away from
   pole = positive y). Now uses north_sign = -1 for north-pole,
   +1 for south-pole, matching sightmap.py exactly.

2. The convergence was subtracted (geo_az - convergence) when it
   should be added (geo_az + convergence), matching sightmap.py's
   convention where convergence rotates from grid north to true
   north.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…odule

- Create scripts/rateLimiters.js exporting apilimiter, authLimiter, computeLimiter
- Remove inline rateLimit() definitions from scripts/server.js
- Remove authLimiter/computeLimiter from the 's' setup object
- Update Users/routes/users.js: import authLimiter directly, replace late-binding wrapper
- Update Users/setup.js: remove router._authLimiter assignment
- Update Utils/routes/utils.js: import computeLimiter directly, replace all 8 late-binding wrappers
- Update Utils/setup.js: remove router._computeLimiter assignment
- Update unit tests to import from the shared module

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…tion

Replace the _localNorthAngle + screen-space rotation approach with a
direct forward geodesic method: compute a destination lat/lng 1° along
the desired azimuth, project it through Leaflet, and draw the line.

This avoids potential compound angle errors and works correctly for
any CRS because it uses Leaflet's own projection pipeline end-to-end
rather than computing a screen-space north-offset angle and rotating.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Both the Earth (USGS SF Hill) and Lunar South Pole (LRO LOLA 4000m)
DEMs are now tiled COGs with deflate compression and overviews.
This ensures consistent behavior with the sightmap COG requirement
and enables fast overview-based reads at any resolution.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…server time sync

- chronice.py: add lunar LSMT support using SPICE et2lst with observer
  longitude; format: LDAY-NNNNNLHH:MM:SS; reverse conversion via iterative
  refinement
- utils.js: pass optional lng param through to chronice.py
- SightlineTool.js: remove TimeUI indicator on mode switch, cancel sweep,
  resweep start, and pan-end; pass lng from observer point for LSMT observers
- SightlineElement.jsx: update global TimeControl when observer time inputs
  are changed (blur/Enter), fixing Mars SOL time not updating the TimeUI
- Lunar ref mission config: add Moon (LSMT) observer with type=lsmt

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…key on observer time

- _getObserverLng: fall back to map center when indicatorLastDragPoint is null
- SightlineElement: hide DEM dropdown when no data options configured
- SightlineElement: add onKeyDown Enter handler on observer time inputs

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…e config descriptions

- Only use _projImageOverlay and viewport clipping when the mission uses
  a custom projected CRS (projection.custom=true). For standard longlat/
  Mercator missions (like Mars), the DEM's projected bounds are in a
  different CRS than the map, causing misplaced overlays.
- Restore Layer-specific DEMs config row and improve field descriptions
  in sightline tool config.json (lost during ShadeTool→SightlineTool rename).
- Add sweepColorRamps and observer type examples to descriptionFull.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…a row, clear default name

- Remove Layer-specific DEMs config row (previously asked to remove)
- Restore detailed descriptions from old ShadeTool config:
  - Sources: documents name/value properties, dropdown usage, kernel path
  - Observers: documents name/value/frame/body, chronos setup path
  - Default Height: full description of height parameter behavior
  - Observer Time Placeholder: documents format string usage
  - Frame/Body fields: proper SPICE reference descriptions
- Remove 'Sightline N' default element name (now empty)

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…instead of UTC

Root cause: chronice lmst→utc returns '2026-05-30T21:36:57.975' (no Z suffix).
The old ShadeTool correctly did: result.replace(' ', 'T') + 'Z'
The new code used a regex chain that failed when milliseconds were present
without a trailing Z, leaving the string timezone-ambiguous. new Date()
then parsed it as local time (UTC-7), adding ~7 hours.

Fix: strip milliseconds then unconditionally append Z, matching the old
ShadeTool approach. The /ZZ$/ → Z guard prevents double-Z if chronice
ever returns a Z-suffixed result in the future.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…t, tab-switch regen, add editable time field

1. Sightmap sun direction: _compute_directions now only applies convergence
   rotation for azimuthal projections (stereo/gnomonic). For cylindrical
   projections (Equidistant Cylindrical, Mercator), grid north = geographic
   north so convergence = 0. Previously applied polar-stereo formula to all
   projected CRS, giving ~90deg rotation on Mars DEM.

2. Observer time 1-second drift: restored _lastConvertedMs pattern from old
   ShadeTool. Saves sub-second precision from observer->UTC conversion and
   re-attaches it in UTC->observer reverse conversion for exact round-trips.

3. Tab-switch regeneration: _onTimeChange now tracks _lastGeneratedTime and
   skips if unchanged, preventing redundant sightmap computation when
   TimeControl re-broadcasts the same time on tab refocus.

4. Editable time field (vstOptionTime): restored from old ShadeTool. Shows
   current end time in configured format (DOY, etc), editable on blur/Enter.
   Parses via utcTimeFormat if configured, else appends Z directly.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
- CSS matches old ShadeTool exactly: full-width centered input, bold 14px,
  color-p0 bg, color-a1-5 text, transparent border that shows color-c on focus
- Clock icon positioned absolute right (pointer-events: none) as in original
- Structure uses flexbetween wrapper matching old jQuery markup
- Mars reference mission utcTimeFormat changed to DOY: '%Y-%j %H:%M:%S'
  giving output like '2026-150 21:36:57' instead of ISO format

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…text non-selectable

1. HorizonProfile.py _grid_convergence: same fix as sightmap.py — only
   apply convergence for azimuthal projections (stereo/gnomonic). For
   cylindrical projections (Mars Equidist. Cylindrical), convergence = 0,
   so horizon terrain profile azimuths are now correct.

2. Visibility chart (.sightlineVisWrap): added user-select: none so
   dragging the timeline scrubber doesn't accidentally highlight text.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…ing/lag

- HorizonProfile.py: compute per-axis pixel scales (px_scale_x, px_scale_y)
  so the march direction accounts for longitude compression at observer
  latitude. For geographic CRS at 38°N, 1° lon ≈ 0.79 × 1° lat in meters;
  without this the march traces wrong physical angles, distorting azimuths.
  Also computes correct per-step physical distance instead of using the
  averaged pixel_scale.

- Crosshair restyled: smaller (8px circle, 5px arms), lime green with
  black borders (box-shadow outline).

- Crosshair converted from raw DOM element to Leaflet DivIcon marker.
  Leaflet handles positioning in its own transform pipeline, eliminating
  the lag that occurred when updating CSS left/top on the move event.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…is open

Small 6px lime green dot with 1px black border, always at 50%/50% of
the map container (CSS-only positioning, no event tracking needed).
Added on make(), removed on destroy().

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Previously the crosshair only corrected its position on the next pan
event. Now _updateCrosshairPosition() is called right after sweepCenter
is stored for both static sightmap and batch/sweep completion.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
- Add SAFE_NAME_RE validation on target, obsRefFrame, obsBody to prevent
  directory traversal via SPICE kernel paths (matches /ll2aerll_bulk).
- Add MAX_TIMES=200 cap on sightmap batch to prevent resource exhaustion.
- Fix E2E test: batch response is a raw JSON array, not { results: [...] }.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Match ll2aerll_bulk pattern: handle child.on('error') and
child.stdin.on('error') to prevent hung responses if Python
fails to start. Add !res.headersSent guards on all response
paths in the close handler.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
After open_dem decimates a large DEM, gt[5] is scaled but ds still has
the original RasterYSize. Using ds.RasterYSize with the decimated gt
produces a wrong mid_lat for geographic CRS pixel scale. Now accepts
dem_rows directly from dem.shape.

Also: encodeURIComponent the chronice lng argument to match the other
CLI args (consistency with unquote() on the Python side).

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
cross(normal, north) yields West, not East. Changed to
cross(north, normal) to match the batch version _sun_azel_batch.
Currently unused at runtime but prevents future bugs.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Removed unused scalar functions that were superseded by vectorized
equivalents: sun_azel_at_cell (replaced by _sun_azel_batch),
is_nodata (replaced by _vectorized_is_nodata), geo_to_pixel (never
called). Also removed the unused ds parameter from open_dem return
value and _compute_bounds signature — ds was only kept alive for
get_pixel_scale which no longer needs it after the dem_rows fix.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
SightlineTool.js: removed showSightlinemapLayers, showSweepLayers,
refreshAllHeatmaps, _nextPow2 — all defined but never called.

SightlineTool_Algorithm.js: removed the entire old client-side
sightline algorithm (sightline, processUp/Down, mask, curveData,
isNoData, compositeResults, calcHeight*, initializeGrids, perOctant)
and their unused imports (jquery, F_, L_, G_). Only
cumulativeVisibility is called externally; all other methods were
from the pre-backend era and superseded by sightmap.py.

SightlineTool_Graphs.js: removed _localNorthAngle, replaced by
the geodesic _destinationPoint + _azimuthEndpoint approach.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Remove overflow:hidden from vstSightlineItem, vstSweepCard, and
vstSweepCardsSection so absolutely-positioned color picker palettes
and color ramp dropdowns are no longer clipped by their parent
containers. Add border-radius to headers directly to preserve
rounded corners.

Bump vstColorPalette z-index from 100 to 10000 to match the
ColorRampPicker popup z-index.

Reorder MULTI_SOURCE_COLORS: yellow -> blue -> red -> green
(swapped blue and red positions).

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
et2lst returns integer (hr, mn, sc) so one lunar second spans ~29 ET
seconds. The old iterative loop converged to ±1 lunar second, giving
~30s UTC precision. Now uses binary search after the coarse loop to
find the exact ET boundary where the second ticks over, narrowing to
<0.5 ET seconds. Result is placed at the midpoint of the lunar
second window for minimal round-trip error.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
The Collapsible panel has overflow:hidden for its open/close
animation, which clips the color picker dropdown. Changed the
palette to position:fixed, computed from the swatch's bounding
rect on click, so it escapes all overflow containers.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Reverts position:fixed approach. Instead overrides overflow to
visible on open Collapsible panels inside sightlineTool via
[data-open] selector, so the color palette can extend past the
panel boundary while keeping overflow:hidden during animations.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…-config

feat: SightlineTool + backend sightmap + polar projection fixes + Numba JIT
tariqksoliman and others added 25 commits June 18, 2026 17:57
…eview-panel

fix: DrawTool Review panel hidden by tool panel backdrop-filter containment
…ll control chars in logs

Fix 2: Add permissions: contents: read to secrets-detection.yaml and pin
actions/checkout to SHA (matching bump-version.yml).

Fix 3: Validate _source parameter entries in geodatasets routes to only
allow alphanumeric characters, underscores, and dots before query
construction.

Fix 4: Simplify sanitizeForLog regex to strip ALL control characters
(0x00-0x1F and 0x7F), closing the newline/carriage-return log injection
gap.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Apply forceAlphaNumUnder per-segment (split by dot) to preserve nested
JSON paths like 'properties.name' while sanitizing each path component.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Update inline copy and test expectations to reflect the new regex that
strips ALL control characters (0x00-0x1F, 0x7F) including newlines,
tabs, and carriage returns.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…feedback

Fix 1A: Rewrite newTag HTML injection in DrawTool_Files.js using jQuery
DOM construction (.attr(), .text(), .append()) instead of raw string
interpolation in $.append().

Fix 1B: Remove inline value attributes from F_.sanitize() in
single-quoted attribute contexts (DrawTool_Templater.js:671,
DrawTool_Editing.js:684). Set values safely via jQuery .val() after DOM
insertion.

Add allowList parameter to Utils.forceAlphaNumUnder() to optionally
preserve specific characters (e.g. dots for nested JSON paths).
Simplify _source validation to use forceAlphaNumUnder(s, ['.']) and
fall back to null if all entries are filtered out.

Revise sanitizeForLog to preserve newlines and tabs for stack trace
readability while now stripping carriage returns (0x0D) for log
injection prevention. Update tests to match.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
The initial Fix 1A only covered the 'add new tag' path. Three additional
rendering paths (existing efolders, folders, tags at lines 910-958) also
used raw string interpolation for tag names. Extract buildTagItemHtml()
helper that uses jQuery DOM construction (.attr(), .text()) to safely
render all tag items.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Add '-' to forceAlphaNumUnder allowList since JSONB property names
commonly contain hyphens (e.g. start-time, image-url).

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…-2-3-4

security: XSS prevention, CI hardening, geodataset validation, log sanitization
…nt clipping

Both the value operator and group operator dropdowns in the Filtering
tool were using standard Dropy.construct + Dropy.init which positions
the dropdown content absolutely within the filter row. When rows are
near the top, the dropdown gets clipped by the header/panel.

Switch to Dropy's globalConstruct option which appends the dropdown
content to <body> as position:fixed, escaping ancestor clipping
entirely while preserving all existing selection behavior.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
The globalConstruct path in dropy.js removes the title from the global
element, so the default title update is a no-op. Manually set the
inline title span to the selected item in both onChange callbacks.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…eanup

- Make globalConstruct a function (closure) so the dropdown regenerates
  with the current selected index on each open, fixing stale highlight.
- Support function-valued globalConstruct in dropy.js (backwards-compatible).
- Remove orphaned _global divs from body when filter rows are cleared.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
- destroy(): remove all body-level _global elements matching the
  filtering ID pattern before removing the panel.
- Clear Filter button: remove per-item _global divs alongside their
  row elements during bulk clear.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
- Open downward if trigger is in top half of viewport, upward if in
  bottom half.
- Close the global dropdown on any scroll event (capture phase catches
  nested scrollable containers like the layer list).
- Empty the _global div on outside-click dismissal so it doesn't
  linger at z-index 10000.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…oesn't negate positioning

The openUp CSS applies transform: translateY(-100%) to the UL, which
shifts it upward. When opening downward, this must be removed so the
list renders below the trigger. For upward, the transform correctly
positions the list above bcr.top.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Reverting to original Filtering.js and dropy.js to replace with a
custom compact grid selector instead.

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Replace both the value operator and group operator Dropy dropdowns in
the Filtering tool with a custom compact jQuery grid selector.

- OpGridSelector renders a trigger button and a fixed-position popup
  grid (4 columns) appended to body, escaping ancestor clipping
- Smart open direction: opens down if trigger is in top half of
  viewport, opens up otherwise
- Closes on scroll (capture phase) and outside click
- Value operator: 4x3 grid of 10 operators (=, !=, in, <, >, <=, >=,
  contains, beginswith, endswith)
- Group operator: 4x1 grid (AND, OR, NAND, NOR)
- Removed all Dropy import and CSS from the Filtering tool

Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
Co-Authored-By: tariq.k.soliman <tariqksoliman@gmail.com>
…-dropdown-clipping

feat: replace Dropy dropdowns with custom compact grid selector in Filtering tool
@tariqksoliman tariqksoliman self-assigned this Jun 29, 2026
@tariqksoliman tariqksoliman added the bug Something isn't working label Jun 29, 2026
@tariqksoliman tariqksoliman merged commit f7051a6 into NASA-AMMOS:development Jun 29, 2026
4 of 7 checks passed
@github-project-automation github-project-automation Bot moved this to Done in MMGIS Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant